Papers with balanced accuracy
Detecting Primary Progressive Aphasia (PPA) from Text: A Benchmarking Study (2026.findings-eacl)
Copied to clipboard
Ghofrane Merhbene, Fabian Lecron, Philippe Fortemps, Bradford C. Dickerson, Mascha Kurpicz-Briki, Neguine Rezaii
| Challenge: | Primary progressive aphasia (PPA) is a neurodegenerative disorder characterized by progressive language deficits as the primary symptom. |
| Approach: | They benchmarked the performance of traditional machine learning models with various feature extraction techniques, transformer-based models, and large language models (LLMs) they found that transformer-Based models exceeded chance-level performance in terms of balanced accuracy, while MLP using MentalBert’s embeddings achieved the highest accuracy. |
| Outcome: | The proposed models outperform chance-level models in terms of balanced accuracy while using MentalBert’s embeddings achieve the highest accuracy. |
Sycophantic Anchors: Localizing and Quantifying User Agreement in Reasoning Models (2026.acl-srw)
Copied to clipboard
| Challenge: | sycophancy is a behavior that infiltrates the chain-of-thought, leading models to generate plausible-sounding justifications for incorrect answers. |
| Approach: | They introduce sycophantic anchors that commit models to user agreement . they find scophancy leaves a stronger mechanistic footprint than correct reasoning . |
| Outcome: | The proposed framework outperforms text-only baselines at high commitment levels and predicts commitment strength from activations. |
Discord Questions: A Computational Approach To Diversity Analysis in News Coverage (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Modern news aggregators do the hard work of organizing the news, but choosing which source to read remains challenging. |
| Approach: | They propose a framework to help readers identify source differences and gain an understanding of news coverage diversity by generating questions with a diverse answer pool and reusing existing methods. |
| Outcome: | The proposed framework improves performance from current question generation methods by 5% and achieves 81% balanced accuracy on a realistic test set. |
STORYSUMM: Evaluating Faithfulness in Story Summarization (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing methods for evaluating abstractive summarization are lacking in faithfulness evaluation. |
| Approach: | They propose a dataset that measures faithfulness of LLM summaries with localized errors and faithfulness labels for evaluation methods. |
| Outcome: | The proposed method does not achieve more than 70% accuracy on this task. |
P-QuASAR: A Unified Probabilistic Framework for Holistic Patent Quality Assessment and Refinement (2026.findings-acl)
Copied to clipboard
| Challenge: | Existing methods for assessing patent quality rely on modular pipelines or generic detectors, resulting in fragmented decisions and limited integration across quality dimensions. |
| Approach: | They propose a probabilistic framework that represents patent specifications as Quality Graphs. |
| Outcome: | The proposed framework outperforms existing methods on 500 patents against seven baselines. |
ACUEval: Fine-grained Hallucination Evaluation and Correction for Abstractive Summarization (2024.findings-acl)
Copied to clipboard
| Challenge: | Recent-proposed evaluation metrics for large language models have a preference-bias . however, such metrics often lack interpretability and only offer a single score . |
| Approach: | They propose a metric that leverages the power of large language models to perform two sub-tasks: decomposing summaries into atomic content units and validating them against the source document. |
| Outcome: | The proposed metric improves faithfulness scores on three summarization evaluation benchmarks by 3% compared to the next-best metric. |
HalluGuard: Evidence-Grounded Small Reasoning Models to Mitigate Hallucinations in Retrieval-Augmented Generation (2026.findings-acl)
Copied to clipboard
| Challenge: | Large Language Models excel at NLP tasks but remain prone to hallucinations . small language models can achieve competitive results in specific tasks . |
| Approach: | They propose a 4B-parameter Small Reasoning Model (SRM) that can be used to classify document-claim pairs as grounded or hallucinated in closed-book, document-grounded settings. |
| Outcome: | The proposed model achieves 84.4% balanced accuracy on the RAGTruth subset of the LLM-AggreFact benchmark, surpassing specialized models, MiniCheck (7B; 84.0%) and Granite Guardian 3.3 (82.2%) Across the benchmark, it reaches 77.1% BAcc, surpasses larger general-purpose LLMs such as GPT-4o (75.9%). |
TOXIFRENCH: Benchmarking and Enhancing Language Models via CoT Fine-Tuning for French Toxicity Detection (2026.findings-acl)
Copied to clipboard
| Challenge: | toxicity detection in French remains underdeveloped due to the lack of culturally relevant, human-annotated, large-scale datasets. |
| Approach: | They propose a method that generalizes French online comments using a semi-automated annotation pipeline that reduces manual labeling to only 10% through high-confidence LLM-based pre-annotation and human verification. |
| Outcome: | The proposed model outperforms GPT-4o and DeepSeek-R1 on the benchmark while maintaining cross-lingual capabilities. |
Re-FRAME the Meeting Summarization SCOPE: Fact-Based Summarization and Personalization via Questions (2025.findings-emnlp)
Copied to clipboard
| Challenge: | FRAME reframes summarization as a semantic enrichment task . SCOPE is a reason-out-loud protocol that has the model build a reasoning trace . |
| Approach: | They propose a modular pipeline that reframes summarization as a semantic enrichment task. |
| Outcome: | The proposed pipeline reduces hallucinations and omissions by 2 out of 5 points . SCOPE improves knowledge fit and goal alignment over prompt-only baselines . |